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waves of data from the National Education Longitudinal Study of 1988 
(NELS:88) , it extends an earlier study that demonstrated positive 
effects of high school restructuring not only on learning but also on 
its equitable distribution by social class. The more recent study 
addresses two questions: Do the positive effects of restructuring 
practices persist throughout the high school years? Which 
organizational attributes contribute to these effects? The study 
examined achievement in mathematics and science at the 8th, 10th, and 
12 grades to answer these questions. Although students learned 
somewhat less in the last 2 than the first 2 years, the positive 
effects were sustained throughout high school. More important, 
particular features of the social, structural, and academic 
organization of high schools explained the restructuring effects 
identified in the earlier study. The study identifies larger 
organizational factors that make some high schools better places in 
which to learn than others. It recommends that educators and 
policymakers shrink the bureaucracy; create smaller places within 
schools (for example, schools-within-schools) ; develop cooperative 
social relations within schools; and emphasize academic pursuits. 
Three tables and six endnotes are included. The appendix explains how 
the measures used in the study were constructed. (LMI) 
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Abstract 

This study investigates how the structure of secondary schools affects 
learning. Using a sample of 9,631 students in 789 U.S. high schoo./ with 
three waves of data from NEDS: 88, it extends an earlier study that demon- 
strated positive effects of high-school restructuring not only on learning 
but also on its equitable distribution by social class. This study address 
two questions: Do the positive effects of restructuring practices persist 
throughout the high-school years? Which organizational attributes contri- 
bute to these effects? We examine ac . avement in mathematics and science 
at the 8th, 10th, and 12th grades to answer there questions. Althoxigh 
students learn somewhat less in the last two than the first two years, the 
positive effects are sustained throughout high school. More important, par- 
ticular features of the social, structural, and academic organization of 
high schools explain the restructuring effects identified in the earlier 
study. This study identifies larger organizational factors that make some 
high schools better places in which to learn than others. 
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Understanding High School Restructuring Effects on the 
Equitable Distribution of Learning in Science and Mathematics 

The call to restructure American education has captured the Imagination 
of reformers across the country, yet it has not produced a coherent agenda 
for changing schools, under a "restructuring" umbrella, proponents have 
compiled a wide and often contradictory list of school reforms (Conley 
1993; Elmore 1990; Murphy and Hallinger 1993; Newmann 1991a). These include 
changes in school governance and management; the content and practice of 
teaching and learning; the organization of teachers' work; the quality of 
relationships between staff, students, and community; even the education of 
teachers and administrators at the post -secondary level. Motivating the 
restructuring movement is a belief that significant changes are impossible 
unless almost every aspect of the U.S. public school system is overhauled. 

Three concerns about the U.S. public school system underlie the move- 
ment: (1) students do not learn enough, especially compared to what their 
counterparts learn in other countries; (2) what students do learn is 
fragmented, rote, and disconnected from the real world, and (3) disadvan- 
taged students learn less than their more advantaged classmates (Newmann 
1991a; Quality Education for Minorities Project, 1990) . Restructuring is 
supposed to address one or more of these concerns, although the mechanisms 
by which it would do so are not well understood (Lee and Smith 199S) . 

In an effort to better understand the possible effects of restructuring, 
we extend the results of a prior study that demonstrates positive effects 
for restructuring practices on achievement gains early in high school, 
particularly for students of low socioeconomic status. We identify organi- 
zational properties associated with these effects for learning in science 
and mathematics throughout the high-school years. Before presenting these 
results, we briefly review the literature on school restructuring, inclu- 
ding the results of our prior study. 

Background 



Reforming High Schools 

A wave of top-down reform, sparked by A Nation at Risk (National 
Commission on Excellence in Education 1983) , aimed to raise high- school 
students' achievement by (a) strengthening formal controls over teaching 
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and learning and (b) tightening the loose connection between administation 
and the technical core. Proponents advocated raising graduation require- 
ments , standardizing classroom practices, and holding teachers accountable 
for their students' performance on standardized tests. 

Though such reforms had some desirable effects, such as increasing the 
number of academic courses taken by low-achieving students (Clune and White 
1992) , they also raised concerns about decreased teacher commitment, satis- 
faction, and performance (Rcaenholtz, 1987, 1991) . Evaluations indicated 
that bureaucratic controls of teaching inhibit the acquisition of analytic 
and higher-order skills (Darling -Hammond and Wise 1985) . Some questioned 
whether higher standards would force low-achieving students to drop out 
without more fundamental changes in policies and practices (Natriello, 
McDill and Pallas 1990) . 

Indirect Evidence on School Restructuring 

Modest implementation . Alhough the restructuring movement is in full 
swing, the evidence that restructuring increases student learning, promotes 
deeper understanding, or reduces social inequities in achievement is 
modest. Though the language of restructuring urges radical school change, 
the reality is far more modest (Berends and King 1994; Conley 1993; Elmore 
1990; Murphy and Hallinger 1993; Newmann 1991a). Schools typically 
implement reforms a little at a time: what Tyack and Cuban (1995) call 
"tinkering toward utopia." Some reforms have successfully nudged schools 
toward more organic organization or classroom practices that promote 
higher-order skills, deeper understanding of subjects, and more authentic 
learning tasks (Lee and Smith 1995, 1996; Newmann, Marks, and Gamoran 
1995) . Model programs, sometimes showcased, are usually held at the margins 
of school life and thus influence core activities nr relationships only 
modestly. Pure models of successful restructuring are rare. 

The changes promoted by restructuring, although they sound new, have a 
long history. Many grow out of tensions between traditional and innovative 
practices that have been played out again nd again in previous reform 
movements (Cuban 1990; Rowan 1990) . As a result, some schools are more 
"restructured" than others, even though they have not been active partici- 
pants in the restructuring movement. That is, they engage in practices that 
are consistent with the movement. Such schools provide a setting for inves- 
tigating (at least indirectly) the effects of restructuring, in that they 
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offer an opportunity to evaluate organizational forms that promote restruc- 
turing goals. We discuss three forms that have been linked empirically to 
restructuring goals: the common academic curriculum, authentic instruction, 
and communal school organization. The first two forms characterize the 
organization of academic work; the last focuses on social relations. 

The common curriculum . Students' academic experiences in most high 
schools are compartmentalized, differentiated, and socially stratified. The 
curriculum is divided into discrete areas that are typically organized by 
department. These units organize subjects into course sequences (i.e., 
tracks), access to which is determined by ability, performance, or aspira- 
tions. Such high schools offer students a wide range of courses within each 
department. Although this type of curriculum provides opportunities to 
explore numerous interests, the courses vary considerably in their academic 
content and expectations for performance (Powell, Farrar, arid Cohen 1985) . 
Expansion of the curriculum both horizontally (in terms of tracks) and ver- 
tically (multiple offerings within tracks or even multiple versions of the 
same course with different expectations) creates substantial differences 
in what students study and learn within the same school (Lee et al. 1993) . 

Disadvantaged students are especially harmed by a highly differentiated 
curriculm. More of their courses are low-track offerings that require less 
academic effort, have lower expectations for achievement, and have less 
high-level content (Oakes 19P5; Sedlak, Wheeler, Pullin, and Cusick 1986) . 

A growing body of research suggests that low- income and minority students 
are especially advantaged in schools with a narrow curriculum and a strong 
academic focus (Lee et al. 1993). Because the courses are more similar in 
academic content and expectations, students in different classrooms have 
similar academic experiences. Although this form of academic organization 
has been associated with Catholic schools, Bryk, Lee, and Holland (1993) 
suggest that public schools with similar structures would also have high 
and more socially equitable achievement. 

Authentic instruction . One conception of how knowledge is acquired 
emphasizes the standardization of teaching practices and learning tasks. 
Evidence is accumulating that this conception fails to provide opportuni- 
ties for students to develop more advanced thinking skills, higher levels 
of proficiency in academic subjects, and a sense of themselves as active 
learners (McCaslin and Good 1992; Newmann 1991a) . A different approach to 
instruction requires that studenta^be involved in constructing (rather than 
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reproducing) knowledge through disciplined and sustained involvement in 
tasks that resemble real-life. problems. 

Although all students benefit from authentic instruction, there is 
evidence that such practices may be particularly beneficial for socially 
disadvantaged students. Such students, however, are especially unlikely to 
experience authentic instruction (Cole and Griffin 1987; Quality Education 
for Minorities Project 1990) . Disadvantaged students, often also low- 
achieving, are typically in classrooms that emphasize lower -order skills, 
basic knowledge, drill and practice, recitation, and desk work (Levine 
1988; Knapp and Shields 1990). However, low-achieving students can master 
more complex and demanding tasks, and these richer learning environments 
can lead to dramatic gains in achievement (AAAS 1984; Knapp and Shields 
1990; Kozma and Croninger 1992; Levine 1988) . 

Communally organized schools . Bureaucratically structured schools rely 
on affectively neutral social relationships to facilitate the administra- 
tion of standardized rules and procedures. Strong personal ties between 
adults, or between adults and students, make it more difficult for staff 
to comply with standard practices and procedures. Yet, as Willard Waller 
(1932) noted long ago, emotional bonds between teachers and students are 
crucial in engaging and motivating students to learn. The quality of 
affective ties between staff also directly influences teacher commitment 
and indirectly affects student achievement (Bryk et al. 1993; Rosenholtz 
1991; Lee and Smith 1996) . The alienating and disengaging qualities of the 
typical high school have received much attention (see Firestone and 
Rosenblum 1988; LeCompte and Dworkin 1991). Restructuring proposals have 
encouraged downsizing schools, creating stronger bonds and more trusting 
relationships between students and adults, and facilitating greater colla- 
boration and cooperation between teachers (e.g., Sizer 1984). 

Although the importance of creating a sense of community in schools is 
generally recognized, a distinct theory and empirical studies documenting 
the advantages of communitarian schools are quite recent.* Rather than 
formal and affectively neutral relationships, in communally organized 
schools members are commit ‘..ed to a common mission. Staff and students 
interact outside the classroom; adults see themselves as responsible for 
students’ total development, not just for the mastery of lessons. Teachers 
share responsibility for students' academic success, exchanging information 
and coordinating efforts between classrooms and across grades. Outcomes are 
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more positive in such schools both for teachers (e.g. satisfaction, morale, 
absenteeism) and for students (e.g., class cutting, absenteeism, and drop- 
ping out) . Attending these more positive and caring schools is especially 
beneficial for disadvantaged students (Bryk and Driscoll 1988; Bryk et al. 
1993; Lee et al. 1993). 

School organization and restructuring. A growing body of educational 
research, mostly sociological in nature, suggests that both the academic 
and social organization of schools influences not only student learning 
(including acquiring more advanced competencies) but also the social dis- 
tribution of learning. Schools that emphasize academics, that present 
students with fewer non-academic curricular offerings, and that encourage 
authentic forms of teaching promote the goals of restructuring. Schools 
that foster more supportive and positive social relationships, both among 
staff and between staff and students, enhance the commitment of teachers 
and the engagement of students. Although these forms of school organization 
may not be a direct result of restructuring, assessing their effects can 
provide indirect evidence that changing the structure of secondary schools 
can make them more effective. We hypothesize that restructuring activities 
in schools with these forms are more successful than those initiated in 
schools without them. 



Research Questions 

Previous work . The study described here is a longitudinal followup to a 
study recently published in this journal, "Effects of High School Restruc- 
turing and Size on Gains in Achievement and Engagement for Early Secondary 
School Students" (Lee and Smith 1995) . The purpose of that study (hereafter 
called the Early Restructuring Study, or ERS) was to assess the impact on 
students, in their early years of high school, of attending schools employ- 
ing practices consistent with the school restructuring reform movement. 
Restructuring effects were evaluated on growth in students' engagement and 
achievement, as well as the social distribution of those gains. Results 
were strong and consistent: achievement and engagement were significantly 
higher in schools classified as "restructuring." Those outcomes were also 
distributed more equitablly in restructuring schools. 

Results from the E: S received considerable attention, including a 
citation in Newsweek (hancock, 1994) as an example of research countering 
the conclusions from Herrnstein and Murray's controversial 1994 book, The 
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Ball Curve . Excitement centered on the findings that (a) high schools with 
certain organizational forms ("small and cooperative") were shown to have 
positive effects on students' learning in their first two years, (b) these 
organizational forms also induced a more equitable distribution of learning 
among students from different social-class backgrounds ("the gap between 
the poor and. those who were not poor shrank in the more nurturing schools," 
(Hancock 1994:61), and (c) students in smaller high schools experienced 
more positive outcomes in learning and equity. 

Expanding on the BRS . Although the BRS findings that particular school 
structures could foster positive effects on both learning and equity were 
gratifying, we needed to understand the meaning of our findings. Two 
questions drove this study: "Do the effects of school restructuring in the 
early years of high school endure?" More important, "What organizational 
characteristics of high schools help to explain the apparent effects of 
restructuring practices?" Because the analyses in the ERS were already 
quite complex, we have restricted our analyses here to achievement growth 
in twc subjects: mathematics and science. 

Method 



Sample. Data, and Analysis 

Sampling design . We drew our sample from the first three waves of the 
National Education Longitudinal Study of 1988 (NELS) , a longitudinal study 
of the educational status and progress of U.S. students and schools spon- 
sored by the National Center for Education Statistics (NCES) . In 1988 NCES 
drew random samples of about 25 eighth- graders in each of about 1,000 
middle-grade schools. Students were traced to high schools in 1990, with 
reasonably high response rates. 

Data filters . This study is the third in a series using NELS data to 
study the effects of school restructuring. The sample includes NELS 12th 
graders who fit these data filters: (1) students had to have full cognitive 
test -score data for the three waves; (2) there must be data from their high 
schools and their teachers; (3) students had to be in public, Catholic, or 
elite private high schools; (4) they must have been attending high schools 
with at least 5 NELS-sampled students in them at grade 10; and (5) they had 
to be in the same high schools at 10th and 12th grade (Ingels et al . 1993a. 
1993b) . We used data filters 1-4 for the ERS (Lee and Smith 1995) . 
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The sample for this study includes 9,570 seniors in 789 high schools 
(all included in the 8RS) . The nested sample averaged 12.2 students /school . 
The large majority of sampled schools were public. Because the original 
NELS sampling design involved oversampling certain types of students and 
schools, we constructed a school -level weight in order to generalize to the 
U.S. population of high school students and schools. NELS did not include 
high school-level design weights in their data files. The procedure for 
constructing the weights is descibed in Lee and Smith (1995) . 

Hierarchical approach . Because the research questions for this study 
focus on how students' learning in mathematics and science is influenced by 
the organization of the high schools they attend, we use an analytic method 
designed for such questions: hierarchical linear models [HLM] (Bryk and 
Raudenbush 1992) . We used HLM in our previous restructuring studies (Lee 
and Smith 1993; 1995). The method, now common in school effect studies, has 
been described elsewhere. The HU1 models in this study have a 3-level 
nested structure: multiple test scores nested in students, which are in 
turn nested in schools. 

The three -level HLM model of change over time is described in Chapter 
6 of Bryk and Raudenbush (1992) . How that technique applies here is some- 
what restrictive, given the modest number of time points to model change. 
The NELS test scores allow the estimation of change in a repeated observa- 
tions model, as each score positions a student on an absolute scale of 
science or math performance . Differences in scores between time points are 
framed as "growth, " with the difference in scores on the same test between 
two time points framed as the student's "gain" in performance. We explored 
two growth parameters: (1) 8th to 10th grade (early], and (2) 10th to 12th 
grade [late] . Of course, the parameters are not independent, nor is either 
independent of initial status (8th grade) . Readers interested in more 
detail about this application should contact the authors directly. 

Classifying School s bv Their Structural Practices 

Logic for grouping school s. The categorization of high schools in this 
study, identical to that in the ERS, was based on the degree to which they 
reported practices consistent with the restructuring reform movement . We 
used three major (and interconnected) criteria: (1) the logic laid out the 
literature reviewed above; (2) the broader literature on school communal 
organization (Bryk and Driscoll 1988; Bryk et al. 1993) ; and (3) def ini- 
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tions developed over several years by the Center on Organization and 
Restructuring of Schools [CORS] (Neumann 1991b) . We categorized schools in 
a two-step process. Step one involved classifying a set of 30 reform 
practices on which data were available on NBLS high schools. Principals 
indicated whether their school currently had each practice in place. In 
step two, we grouped the NELS high schools according to the number and 
type of reform practices which principals reported were currently in place. 
Our emphasis was not on whether a practice was new, but on whether it was 
consistent with practices advocated by the restructuring movement. 

Grouping reform practices . We grouped the 30 reform practices as "re- 
structuring" or "traditional" according to these criteria. Our definition 
of school restructuring, which is consistent with research and theory 
developed by CORS, combines two ideas: (1) reforms moving the school away 
from bureaucratic structures and (2) reforms that depart from conventional 
practice. Reforms classified as "Restructuring Practices" fit both ideas: 
they moved schools away from a bureaucratic structure and they were also 
less common. 

Classifying schools . Once the reforms were classified as traditional 
or restructured, we investigated the number and type of reforms in place in 
each school. Schools were unlikely to engage in reforms along a singular 
dimension (such as instruction, authority, or social relations) , nor did 
they typically engage exclusively in restructuring practices. Rather, 
schools were likely to report engaging in reforms of different types simul- 
taneously, consistent with the incremental model of change described above. 
We classified schools engaging in at least three restructuring practices as 
restructured schools . ^ Schools that engaged in several traditional reforms 
but fewer than three restructuring reforms were classified as traditional 
practice schools. A small proportion of schools reported engaging in none 
of the 30 practices, and were classified as no reform schools. More detail 
on item wording for each reform practice, its grouping as traditional or 
restructured, the probabilitiy of its occurrence in NELS high schools, and 
the logic underlying the grouping of reform practices and categorization 
of schools according to these practices is found in Lee and Smith (1995) . 

Measures 

Outcomes . This study examines school effects on achievement jrowth in 
mathematics and science. Three rationales guided our choice of outcomes: 
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(1) we restricted the number of NELS subject areas to limit the complexity 
of results; (2) we selected math and science because data on classroom 
instruction collected by NELS was limited to these subjects; and (3) 

NELS information on students course taking in these subjects was much more 
precise than in other subject, both from self-reports and from transcripts. 
Actually, we investigated four outcomes for each subject: early and late 
achievement gain (Outcomes 1 and 2) and the social distribution of these 
early and late gains according to students' social class [SES] (Outcomes 3 
and 4). These outcomes are displayed schematically in Figures 1 and 2. 
Figure 1 displays Outcomes 1 and 2: early (8th- 10th) and late (10th-12th) 
achievement growth in math and science . Results are reported in standard 
deviation (SD) units to facilitate comparison across tests with different 
numbers of items . It is clear that although students gained knowledge in 
both subjects over both periods, they learned more early (Outcome 1) than 
late (Outcome 2) in high school. Figure 2 represents Outcomes 3 and 4, 
which represent the social distribution of achievement gain in mathematics 
(A) and science (B) within each school. 



Insert Figures 1 and 2 about here 



Three trends are relevant. First, students learn more earlier than 
later high school in both subjects (i.e., the slopes are steeper for Out- 
come 1 than 2) . Second, students don't learn very much over either period 
(i.e., the learning slopes are not very steep) . Third, the learning rates 
are not constant across subjects: students learn more mathematics than 
science, particularly in the early high school years. Using terminology 
suggested by Bryk and Raudenbush (1992) , Outcomes 1 and 2 are effectiveness 
parameters in our HLM models. 

Figure 2 displays effects computed from within-school HLMs . Displayed 
graphically is the relationship between SES and achievement gains in mathe- 
matics (Panel A of Figure 2) and science (panel B) over these two periods, 
the equity parameters. Outcome 3 represents the SES slope on math and 
science learning early in uxgh school, whereas Outcome 4 is the SES slope 
on learning in the last two years. Figure 2 suggests that the relationship 
(or slope) between SES and gain in both subjects varies over time and over 
SES level. While the slope decreases over time, it is steeper for higher- 
SES students. Outcomes 1 and 3 were explored in the ERS study. 
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Independent variables . Independent variables describe both students and 
schools. Our major focus is on the latter group. More detail on construc- 
tion of some variables are found in the Appendix. Because variables used 
in the EPS study are described elsewhere (Lee and Smith 1995) , the Appendix 
provides details only on measures that are new to this study. We describe 
all control variables used in the study, however. 

Controls on students. Within- school controls are of three types: 
demographic characteristics, academic status at high-school entry, and 
course taking. Demographic controls include: SES (a z-score with mean [M] ■ 

0, SD » 1) ; a dummy-coded measure of minority status (Hispanic or Black = 

1, non-minority - 0) ; and gender (female * 1, male ■ 0) . Two measures 
control for status at the beginning of high school: achievement (in either 
math or science, depending on the outcome) and engagement with academic 
activities. We also include controls for ‘students ' course taking patterns 
in mathematics (self-reports of courses in the first two years of high 
school for Outcome 1; a measure of all high-school courses in math and 
science taken from students' transcripts for "odeling Outcome 2) . These 
controls were also included in the ERS. 

Variables describing schools. Between- school controls measuring school 
organization are of four types. The first type, composition and structure, 
includes average SES; minority concentration (a dummy-coded measure with 40 
percent or more minority enrollment coded 1, otherwise 0) ; school sector 
(two dummy variables for Catholic and elite private schools, each compared 
to public schools) ,- and school size. The second type, restructuring status, 
also captures school structure. Based on the school categorizations 
described above, we include two dummy-coded contrasts: (1) restructuring 
schools compared to traditional schools, and (2) schools without reform 
practice compared to traditional practice schools. Although school 
composition and structure measures are used as statistical controls, the 
restructuring contrasts represent a central construct in the study. 

Variables of the third type tap school social organization. Although 
we investigated several such measures, our HLM models include a single 
■ariable: collective responsibility for learning. This composite of 
items capturing teachers' attitudes about their personal willingness and 
ability to alter teaching methods to respond to the learning difficulties 
of their students was shown elsewhere to have strong effects on student 
learning (Lee and Smith 1996 ) . 
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School measures of the fourth type tap the construct of academic 
organization. One variable, the school average of students' course taking 
in academic mathematics and science courses, is meant as a proxy measure of 
the common curriculum. We also used the sta n da r d deviation of course taking 
to indicate variability in students' intellectual experiences. Academic 
press is a composite of principals' reports about the importance the school 
places on academic pursuits and the morale of teachers and students . 

Although morale could be high for other than academic reasons, the strong 
reliability of this composite suggests a colescence among school members 
around an academic mission for the school. 

Also capturing elements of schools' academic organization are two 
measures that focus on instruction. One, authentic instruction in science 
and mathematics, is the mean of four school-level aggregates of students' 
and teachers' reports of the frequency of various instructional activities 
in those two subjects. 3 Rather than factor analyis, the construction of 
this variable was with a technique, Rasch modeling, more commonly used to 
rescale test score items or rating scales. Rasch scaling is based on an 
accumulation of individual instruction practices, rather than the use of 
certain practices in place of others (Rasch 1980) . We use the composite as 
a measure of instructionally rich classrooms that use multiple practices 
associated with constructivist teaching or active learning; what Newmann 
labels authentic pedagogy (Newmann 1991b; Newmann, Marks, and Gamoran 
1995) . Another variable, the standard deviation of the Rasch scales from 
student items, taps variability in authentic instruction within a school. 

In order to understand how high scnool organization influences learning 
and its equitable distribution, and to gauge the relationship between 
school organization and school restructuring, we first investigate how 
these school characteristics are related to the categorication of schools 
by restructuring practices. 

Character ist ice of Schools in the Restructuring Categories 

School characteristics . In Table 1, descriptive information on 
the characteristics of the 789 schools in the sample is broken down by the 
schools grouped as those without reform practices (column 1) , those with 
traditional practices (column 2), and schools with restructuring practices 
(column 3) . Group means were computed and tested using one-way ANOVA with 
two contrasts: (1) rr -reform compared to traditional schools, and (2) 
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restructuring practice schools compared to traditional schools. Almost half 
(46 percent) are restructuring practice schools, 43 percent are traditional 
practice schools, and 11 percent are schools without reform practices. The 
schools' organizational characteristics are distinctive in several ways. 



Insert Table 1 about here 



Composition and structure. Schools without reform practices are signi- 
ficantly disadvantaged compared to traditional practice schools on several 
social demographic factors (more minority students and students of lower 
ability). More striking is the advantage of restructuring practice schools 
compared to the same group in terms of average SES and average 8th grade 
achievement. Distribution by sector explains some of the social background 
differences. Private schools represent 14 percent of the school sample, the 
majority of which are restructuring practice schools. Nevertheless, the 
overwhelming majority of all groups are public schools. Traditional schools 
are smaller than father restructuring or no-reform schools. 

Social organization. School social organization focuses on reports 
from teachers about how much responsibility they take for the learning of 
their students. The pattern of group differences that favors restructured 
schools suggests that social organization may be intertwined with school 
restructuring, although the causal direction of relationships is unclear. 

Academic organization. A similar pattern is evident among measures of 
academic organization. Restructuring practice schools are significantly 
advantaged on all these measures: students take more math and science 
courses and there is less variability in course taking, instruction is more 
authentic, authentic instruction is more homogeneous across classes in 
restructuring practice schools, and these schools have higher levels of 
academic press. Although again the causal direction is unclear, the pattern 
is not; restructuring schools have stronger academic organizations. 

The patterns from Table 1 are distinct. Restructured schools, are 
advantaged in the types of students who go to them, including (and not 
independent of) the larger proportion of private schools in their ranks. 
Therefore, the demographic and structural organization of zchoois should be 
taken into account in any analyses that compare these schools, above and 
beyond controlling for che types of students who go to the schools. It is 
evident that restructuring practice schools are also advantaged in terms of 
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organizational characteristics. Thus, these factors are probably important 
in understanding any restructuring effects on student learning. 

Resul t s 

Revisiting Restructuring Effects Early in High School 

Summary of the ERS , As stated, the ERS study evaluated the effects of 
school restructuring, based on the categorization of schools we described, 
on achievement gains over the early high school years (Lee and Smith 1995) . 
The pattern of effects was clear and consistent: students learned more in 
mathematics, reading, history, and science in restructuring schools 
compared to traditional schools (Outcome 1 in Figure 1) . Effect sizes (ES) 
of the "restructuring school advantage" on learning averaged between .35 
and .59 SD, moderate to large effects. Equally important, achievement gains 
in all subjects for students attending restructuring schools were more 
equitably distributed by student SES (effects on Outcome 3 averaged about 
-.3 SD) . Students in schools without reform practices were disadvantaged on 
these outcomes compared to their counterparts in traditional schools, 
although ESs were somewhat smaller. Results were estimated in HLM models 
that included within- school controls for students' minority status, gender, 
SES, 8th-grade ability, and 8th grade engagement. Between- school controls 
included average school SES, minority concentration, school size, sector, 
average course taking, and variability in course taking. 

Revisiting the research questions . But what do the consistent and 
encouraging results of school restructuring on learning early in high 
school really mean? Because we know that learning builds on prior 
learning, we need an analytic model that takes early gains into account -- 
one that considers a student's learning trajectory rather than simple 
status changes investigated so far. First, we want to know whether the 
effects of attending schools classified by their restructuring status are 
sustained on learning later in high school, with a model that takes early 
learning into account. More important is to understand the implications of 
the restructuring effects we have shown. Thus, the second research question 
asks, "Are there other organizational characteristics of these high schools 
that would help us understand why schools classified by restructuring prac- 
tices have such favorable effects on their students?" As stated, we limited 
these analyses to learning in science and mathematics. 
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Organizational Effects on Early and Late Achievement Gains 

Within-school (level 2) HLM model . NELS measured students' achievement 
at three important time points: near the end of 8th grade (entry into high 
school) ; in loth grade (midway through high school) ; and at the end of 12th 
grade (just before graduation) . Ke stuctured our HLM analyses as described 
in the Methods section. Figures 1 and 2 indicated the four outcomes we 
investigated in the two subject. The two gains (early and late) in science 
and mathematics achievement we call "Outcome 1" (from 8th to 10 grace) and 
"Outcome 2" (be tv sen 10th to 12th grade). Outcomes 3 and 4 focus on the 
relationship between SES and those gains . The distributions of Outcome s 1 
and 2 are described in the top panel of Table 2 . 



Insert Table 2 about here 



The average early gain on the 25-item science test is larger (2.86 
points) than the later gain (1.60) . Tha average early gain on the 40-item 
math test (8.47) is larger than the later gain (4.66). Students learn 
little science over the course of high school, at least as measured by NELS 
test items; they learn somewhat more mathematics. 4 Although SDs of the two 
science gains are similar (and large) , the SDs of the math gains differ ; 
the late gain is less variable. HLM variance estimates of within- school 
gains in both subjects, pooled across schools (tau-pi) , are much larger 
than between-school variances (tau-beta) . In a 3-level HLM model, most of 
the variation in the outcome is captured by the measurement (Level -1) 
model. All four gains are quite reliable (.85-. 95), as extimated *-y HLM, 
despite modest between-school variances. 

The Level-2 (within-school) HLM model for the learning outcomes (early 
and late gain in science and math) are shown in the bottom panel of Table 
2. As before, we present effects of student characteristics on on Outcomes 
1 and 2, estimated simultaneously for each subject, in effect size (ES) 
units, computed by dividing the estimated gamma coefficients by the SD of 
each outcome estimated with a fully unconditional HLM. We remind readers 
that the HLM growth model includes control for early gain on late gain, as 
well as a control for ability. Several patterns are evident. While engage- 
ment has no effect on learning, prior achievement (measured at 8th grade in 
each subject) has a substantial effect. In both subjects, effects are very 
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large on early gains (BSs over 2 SD) . Other student characteristics exert 
quite different effects for the two subjects. For example, 8th-grade 
achievement has no effect on late gain in science, but it is related to 
late gains in math (ES - .9 SD) . The number of academic courses taken in 
math and science early in high school has no effect on early science gain, 
but is an important predictor of early math learning (ES • .5 SD) . We note 
that courses taken in these subjects in the last half of high school exert 
no residual effect on late gain, once other variables are controlled. 

Demographic effects differ by subject. Social class (SES) has stronger 
effects on late than early gains in both subjects, particularly in 
mathematics (ES - 1.2 SD for late gain, compared to .7 SD on early gain) . 

The SES effect on science gain is more similar for the two gain measures 
(ES - .6 SD on early gain, .7 SD on late gain) . Gender effects are large 
and favor males, especially in science (ES- -1.4 SD for both time spans). 
Although the gender effect also favors males in math, it is much larger for 
late (ES- -.8 SD) than early gain (ES - -.2 SD) . The effects of minority 
status are unusual. For early gains in both math and science, minority 
status is negatively related, although the effect is much large on early 
science gain (ES- -1.3 SD) than early math gain (ES- -.6 SD) . Minority 
effects on late gains are positive, however, and they also increase over 
time. Many of these effects are important and interesting; however, because 
they do not relate to the research questions in this study we discuss them 
no further. Their magnitudes suggest the importance of including them as 
statistical controls. The SES effects in Table 2 represent Outcomes 3 and 4 
in subsequent analyses (see Figure 2) . 

Are the Effects of School Restructuring Sustained ? 

Between- school (Level 3) HLM restructuring model. Using the within - 
school HLM models shown in Table 2, we estimated a set of 3 -level HLM 
analyses on outcomes 1 to 4 with a a growth modeling format. Although the 
intent of this analysis is similar to the ERS described earlier, it differs 
from that study in two important respects: (1) it uses a growth trajectory 
design, whereas the ERS used a simpler 2-level HLM design that used test 
score differences as outcomes; and (2) it compares restructuring effects on 
gains early and late in high school. Besides the statistical controls on 
students, the models also include the following school -level controls: 
average school SES, minority concentration, school sector, and school size. 
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Results, shown in Table 3, are also resented in ES units. In general, the 
results confirm the stability of restructuring effects, over time and over 
subject matter. 



Insert Table 3 about here 



For Outcomes 1 and 2 (the effectiveness parameters ) , the learning of 
students attending restructuring practice jhools was significantly higher 
in both subjects. In fact, effects increased later in high school (in 
science, ES-.7 SD for Outcome 1, 1.5 SD for Outcome 2; in math, ES-.7 SD 
for Outcome 1, 1.1 SD for Outcome 2) . Students in no-reform schools are 
disadvantaged in learning in both subjects. On the equity parameters 
(Outcomes 3 and 4) , learning in science and math is more equitable among 
students in restructuring practice schools; again the restructuring effects 
increase later in high school. Effects of attending schools without reforms 
suggest that learning in both subjects is both lower and also more socially 
inequitable. Our results here confirm and expand the positive findings from 
the ERS and provide a positive answer to our first research question. Quite 
simply, attending schools that are categorized as restructuring according 
to the definition laid out above has sustained, positive, and equitable 
effects on students' learning in science md mathematics. 

Understanding Restructuring Effects Through Organizatio nal Factors 

Technically, our method of addressing the third research question 
involves an attempt to "explain away" the restructuring effects by inclu- 
ding characteristics of the schools' academic and social organization in 
our HLM models. The results of the HLM organizational models that answer 
this question are displayed in Table 7. 5 It is evident that organizational 
effects on the four outcomes (early and late achievement gains in science 
and mathematics .^nd the SES slopes on each of these gains) are large, con- 
sistent, and ‘ ;gical. We organize our discussion of effects around the 
constructs the independent variables represent rather than by outcomes. 



Insert Table 7 about here 



Demographic and structural effects . The effects of both social compo- 
sition (average school SES, minority concentration) and school sector are 
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generally non- significant. However, even after talcing into account several 
related organi rational characteristics of schools, the effects on both the 
effectiveness and the equity parameters of school size are large. Learning 
in science and mathematics is higher in smaller schools, both early and 
late in high school. Even larger effects accrue on the equity parameters; 
smaller schools are more equitable. The size effects are somewhat,* larger on 



late tuan early gains. 
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Restructuring effects . In the organizational models, restructuring 
effects, which were statistically significant in Table 3, have generally 
moved to non- significance. However, the pattern of restructuring effects is 
in the same direction: attending restructuring practice schools has a 
positive effect on the effectiveness parameters, a negative effect on the 
equity parameters. These results provide allow us to respond positively to 
the second research question. That is, the organizational characteristics 
of schools we consider have "explained away" the restructuring effects 
demonstrated in the ERS and in Table 3 of this study. Of course, this 
finding provides only a first-cut answer to the original question. We 
would like to know which organizational effect matter. 

Social organization . School social organization, measured here as 
collective responsibility for learning, is strongly and positively associ- 
ated with both effectiveness and equity in learning in both subjects, with 
effect sizes in the moderate to large range. 6 Its effects, although 
consistently positive, differ over time and subject. The effect of collec- 
tive responsibility on science learning increases in magnitude from the 
beginning to the end of high school (from .39 SD to 1.0 SD) , but declines 
somewhat for math learning (from .89 SD to .40 SD) . Effects on equity are 
large and increase over time in both subjects. 

Academic organization . Our strongest organizational effects on both 
learning and its equitable distribution in science and mathematics are, not 
surprisingly, associated with schools' academic organization. We discuss 
measures of the average levels of academic course taking in mathematics and 
science and the variability in course taking as a set. We interpret them as 
a combined indicator of a school that offers a narrow curriculum that is 
academic in content, one where most students take the same courses of this 
type (i.e, variability in course taking is low) . Results here indicate, 
quite consistently, that in such "core curriculum" schools, students learn 
more, and learning is more equitably distributed. A general pattern is 
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larger effects in math than science for early gains, but larger in math 
than science for later gains. 

The measures of instruction in a school, the levfel and distribution of 
authentic instruction, should also be considered together. On average, 
students attending schools that are instructionally rich and incorporate 
active learning, and where this type of instruction is shared widely (i.e., 
the variability of authentic instruction is constrained) , gain more in 
science and math achievement both early and late in high school. Less 
variability in authentic instruction is also associated with gains that are 
more equitably distributed. A similar pattern of timing and subject matter 
is evident for both authentic instruction and curriculum structure. 

For these variable pairs measuring school academic organization, there 
is an important pattern of effects. The school average of both measures 
(academic coursetaking, authentic instruction) are more strongly related to 
the effectiveness parameters (i.e, learning) , whereas the variability in 
these measures shows stronger association with equity parameters (the SES 
slopes on learning) . That distributional measures are associated with 
distributional outcomes, and aggregate measures are related to aggregate 
outcomes, makes good sense but is also substantively important. 

Yet another measure of academic organization, school academic press, 
follows the same general pattern of effects as the other measures of this 
construct. Academic press is positively associated with both early and late 
learning, and is negatively associated with the SES/gain slopes. The 
effects here are subject-specific (at least in terms of statistical signi- 
ficance) : in schools with mo.e academic press, students learn significantly 
more in science and that learning is more equitably distributed. 

Discussion 

School Restructuring and School Organization 

Organization counts . The results of our two studies of the influence 
of high-school restructuring on students' learning yield consistent conclu- 
sions. One important and obvious finding is that resturucturing practices, 
as we have identified them, really matter. Another is that some attributes 
of school organization may help to explain the success of recent efforts to 
restructure schools. 

Effects endure . One important finding resulted from trying to "unpack" 



Explaining Restructuring Effects on Achievement Gains 

19 

the notion of school restructuring by targeting organizational features 
that help us understand the effects we had shown in the ERS. The restruct- 
uring categories we considered were meant to capture schools' willingness 
to adopt and stick to reforms that moved them away from bureaucracies and 
toward communities. Rather than disappearing after the early years of high 
school, when we might expect students to be most influenced by schools that 
are relatively new to them, effects on learning persisted or increased 
somewhat in the latter years of high school. 

Distributions are important . The outcomes we considered incorporated 
the joint notions of effectiveness and equity which we have used to define 
"good" schools. Although the general trends of effects for our academic and 
social organization measures showed favorable effects on both, the pattern 
was specific. For both the common curriculum and authentic instruction 
constructs, levels of course taking or authentic instruction in each school 
are associated with average learning, whereas the pervasiveness of such 
practices among school members is associated more with social equity in 
learning. Small school size, although related to both effectiveness and 
equity, is most strongly associated with the distribution of learning. This 
pattern of association points out the importance of considering both levels 
and distributions as outcomes, when evaluating organizational effects. 

Organization. Learn ing, and Community 

Shrink the bureaucracy . We revisit here arguments we have made 
elsewhere that focus or the contrast between schools as bureaucracies or 
communities. Scholars agrea about the dominance of the bureaucratic model 
of U.S. public secondary schools. Historically, there were valid reasons to 
structure schools in that way. Even recently, increased technological and 
human resources , coupled with modern management techniques , have been seen 
as the most appropriate means to foster student achievement. The modern 
bureaucracy seemed to be the best way to provide equal educational opportu- 
nities to students disadvantaged by race, poverty, or immigrant status. 
Through expanded curricular offerings and specialized teachers' work, high 
schools aimed to become universal institutions that offered something for 
everyone. But such idealistic aims do not seem to have been realized. Our 
two studies of school restructuring give hints about this failure. 

Create smaller places . Our findings support a move to smaller high 
schools, without new bricks, mortar, bond issues, or millage increases, a 
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reasonable approach would be to create schools-wi thin- schools: smaller 
organizational units inside the existing walls of large high schools. Here, 
teachers and students would know one another, and school members would see 
themselves as part of a "school family." We suggest that small size is not 
an end in itself, but rather acts as a facilitating or inhibiting factor. 

It is likely to facilitiate more personalized social interactions. But it 
inhibits a differentiated curriculum and teacher specialization (major 
features of bureaucratic high schools) . Recent reforms in New York City 
suggest that this idea has caught hold, at least in that venue (Dillon and 
Berger 1995) . Once high schools are restruct’ rsd into smaller units, what 
might we see if we walk inside? 

a n«w type of social relations . A different sort of social interactions 
than is typical in today's high schools would predominate: less hierarchy, 

e 

less specialization, more cooperation. Our results suggest that teachers in 
"good" schools believe that they can (or should) succeed with all of their 
students. Rather than laying the cause of academic problems on the students 
and their families (i.e., outside the schools and outside themselves), such 
teachers take responsibility for correcting them. The measure, in collec- 
tive terms, surely taps a pervasive type of social relations. We find 
McLaughlin’s (1994:11) comment compelling: "...personalized school environ- 
ments, settings where teachers and students can come to know one another, 
and where students feel acknowledged and respected as individuals . " What 
kinds of activities would we find students involved with in such schools? 

Academic pursuits predominate . The communal perspective is more con- 
cerned with affective than cognitive dimensions of schooling. But evidence 
suggests that "good" schools also have a strong academic structure. Rather 
than a broad range of courses at many different levels, rather than many 
students selecting courses according to their "personal tastes" (the uni- 
versalistic model) , our evidence supports the positive value of a narrow 
and academic curriculum, with a strong orgnizational push for all students 
to take (and master) these courses. Within the courses, we have evidence 
that instruction should include more authentic practices, and that this 
type of teaching should be pervasive rather than restricted to classi ooms 
wher teachers happen to prefer teaching this way. In commenting on th 
ERS, Bryk (1994:7) highlighted our "evidence that a constrained academic 
structure in high schools plays a key role in the equitable social 
distribution of achievement." 
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These studies seem to offer support for a body of sociological research 
that school organization matters, and that the optimal organizational form 
for high schools is more communal than bureaucratic. Although convinced 
that this vision of the American high school would improve student learning 
also its equitable distribution, we are far from sanguine about the 
best way to accomplish what will is a major organi rational shift. School 
restructuring as a "reform movement" is difficult to conceptualize. Much 
of the term's appeal lays in its vagueness, so that people may "see" in it 
whatever they wish to see (Newmann 1991a) . Our results may clear up some 
of the vagueness , making the direction for change clearer . 
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1. Communitarian models of school are more complex than we describe here. 
Their features are not limited to the social relations that characterize 
interactions between staff or staff and students. A fuller description 
and e x a min ation of the effects of school professional community can be 
found in work of Louis and her colleagues (Louis 1990; Louis, Kruse, and 
Associates 1995; Louis, Marks, and Kruse 1994) . The effects of communal 
organizational forms on students are shown in Bryk and Driscoll, 1988; 
Bryk et al. 1993; and Lee et al. 1993) . 

2. The decision to use three restructuring practices as a cutpoint, which 
may seem arbitrary at first glance, was confirmed with a series of 
sensitivity analyses in the ERS (see Figures 1 and 2 [pp. 260-261] in 
Lee and Smith 1995 for details) . Learning was greatest, and its 
distribution most equitable, in schools with 3 or 4 reform practices 
classified as restructuring. 

3. We have two reasons for combining data about mathematics and science 

instruction. One reason concerns the sample: we want to include the 

entire sample of students. In both the base year and followups, NELS 
collected data from two of each student's teachers -- either English or 
social studies, and either mathematics or science. Thus, students do not 
have data from both science and math teachers, but rather from one or 
the other. The second reason has to do with data quality. Including 
data from both students and teachers about instruction in their schools 
produces more reliable measures. 

4. As Ingels et al. (1993a) explain, the NELS math tests were "tailored". 
That is, at the first follow-up, initially lower scoring students were 
given a set of simpler math items than those with moderate or high 
scores at the base year. This tailoring was introduced to make the tests 
more responsive to effects of the high school mathematics curriculum 
(which is typically tracked by ability) , to avoid ceiling effects for 
the most able students, and to locate the discriminating power of the 
test differentially by student abilicy. The NELS science test was not 
tailored, although more difficult items were introduced and simpler 
items dropped over the three testing periods. Tailoring might explain 
the larger proportional gains in mathematics than science. 

5. Although results on the four outcomes are presented in separate columns, 
it is important to note that all the effects in Table 4 were estimated 
in a single HLM model for each subject. The model was, in fact, even 
more complex -- it included the within- school controls shown in Table 2 . 
But those effects did not change much, nor were they the focus of our 
analyses, so we did not include them in Tables 3 or 4 . 

6. Another measure of school social organization, staff cooperation, was 
statistically significant in an HLM model without the responsibility for 
learning measure. Once the latter variable was introduced, staff 
cooperation dropped to non-: ignificance and was deleted from the model. 
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Table 1: Organisational Characteristics of High Schools With Mo Reform 
Practices, With Traditional or Moderate Practices, and With 
Restructured Practices (n ■ 789 schools) 



Schools Without 

Variable (a) Reform Practices 



Schools With 
Traditional 
Practices 



Schools With 
Restructured 
Practices 



(n-88) 



(n-338) 



(n-363) (b) 



A. Compositio nal and Structural Organisation 
Average SES (c) 

% Minority Enr. 

% Public 
% Catholic 
% NAIS 

Average Ach . , 8 th ( c ) 

School Size 



- . 43- 


-.19 


.33*** 


34.14*** 


10.73 


13.78 


98,80 


96.01 


83.02*** 


0 . i \5~ 
0.3.. 


3.50 


9.73*** 


0.48 


7.25*** 


- . S 2 *V. *♦ 


-.16 


.30*** 


1,091* A 


632 


769** 



B. Social Org anization 
Col-ective Responsibility 

for Learning (c) -.74*** 






C. Academic Organization 
Average Number of Math & Science 
Courses, 9 ->12 4.52 J 

Variability in Math, Science 

Course Taking (SD) 2.05 £ 

Authentic Instructional Practical, 



Sci.a Math (c,d) 
Variability in Authentic 
Inst .Practices (SD) (c) 
Academic Press (c) 



a 

1.63***$ 

( 

.56** | 
- , 25 t 

t 



4.73 

2.00 

- .58 

.13 

-.21 



.31*** 

5.14*** 

1.83*** 

1.04*** 

- .26*** 
.31*** 



~ p < .10; * p £ .05; ** p < |U; *** P < -031 

$ 

a. Group mean differences tester with one-way ANOVA and contrasts. Both 
"No Reform Practices" and "Restructured Practices" schools were 
contrasted (separately) with t'Traditional Practice" schools. 

b. Sample sizes reported on this? table are unweighted. Group means are 
weighted, using the NELS constructed school weights. 

c. These variables are z-scored, with M»0, SD»1. 

d. Created with Rasch-model scaling, and then z-scored. 
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Table 2: HLM Within- School Model for Early and Late Achievement Gains in 
Science (N-9,63l Students in 789 Schools) 





Dependent Variables 




(Outcome 1) 


(Outcome 2) 


Early Gain 


(Gr. 8 -> 10) 


Late Gain 


(Gr. 10 -> 12) 


SCZEKCS 


MATHEMATICS 


8CXEKCS 


MATHEMATICS 


A. Descriptive Information 
Mean Between- 

School Gain 2.86 

Standard Deviation 


8.47 


1.60 


4.66 


of Gain 3.79 


6.33 


3.72 


4.66 


Between- Student 

Variability (Tau-pi) 18.89 
Between- School 


40.13 


11.54 


21.71 


Variability (Tau-beta) .19 


3.90 


.02 


1.00 


HLM Reliability .904 


.953 


.852 


.916 


D. HLM Within-School Model Results 






Fixed Effect (a) 

Estimated Intercept 4.002*** 


8.903*** 


1.981*** 


4.816*** 


(Base) 








Independent Variables (b) 

8th Grade Engagement -.138 


- .352* 


.061 


- .099 


8th Gr .Achievement (c) 2.402**» 


2.326*** 


- .202 


.936** 


Academic Coursetaking in Math 
and Science, 8-10th .083 


.509** 






HS Academic Coursetaking in 
Math and Science, 10-12th 




.301 


.169 


Social Class (d) .607*** 


.745 


. 736*** 


1.213*** 


Minority Status -1.312* 


- .561 


.632 


1.588** 


Gender (Female) -1 . 378*** 


- .189 


-1.411* 


- .797* 


* p < .05; ** p < .01; *** p 


< .001 







a. In these HLM models, the fixed effejts of 8th-grade achievement scores 
are: Science, 19.799***; Mathematics, 37.538***. 

b. All effects are presented are in a standardized effect-size metric, 
computed by dividing the HLM gamma coefficient for each outcome by the 
adjusted HLM school -level standard deviation (SO) of that outcome. 

c. The 8th-grade ability control is constructed as a composite test scores 
at that grade in the three curricular areas not measured by each gain 
score . 

d. In these HLM models, SES is allowed to vary randomly between schools, 
while the other controls are employed as fixed parameters. SES is 
centered around the sample mean, while the other controls are centered 
around their respective school means. 
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Table 3: HU Between- School Restructuring Model for Barly and Late Achievement Sains in Science and 

Mathematics (N-9,63l Students in 789 Schools) 
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(a) 












Outcome 1 
Early Gain 
Grade 8- >10 


Outcome 3 
SES- Slope on 
8->10 Gain 


Outcome 2 
Late Gain 
Grade 10- >12 


Outcome 4 
SES -Slope on 
10- >12 Gain 




Science Math 


Science 


Math 


Science 


Math 


Seiance 


Math 


Base Estimate (b) 


3.86*** 9.12*** 


.67*** 


*..14** 


1 .90**< 


► 4.62*** 


.27 


1.51*** 


A. Demographic and Structural gharacteristlcs (c) 












Average SES 


.49 .97- 


.86 


.12 


.67 


1.28* 


.56 


1.26 


Hi -Minority Enrl. 


.48 -.39 


. 90 


-1.34 


1.06 


-1.87* 


1.03 


-2.47 


Catholic School 


.31 .79 


-1.13 


- .46 


.16 


.80 


- .34 


-1.10 


NAIS School 


-.27 -1.29 


.89 


-3.38- 


- .41 


- .23 


- .52 


4.39 


School Size 


-.56** -.95** 


1.75*** 


1.30** 


-1.33** 


* -.96* 


2.02* 


2.33* 


B. Restructuring Effects (c) 














Restructing v. 
Trad. School 


.69* .69* 


-2.75** 


-1.45** 


1.53* 


1.11** 


-3.26*** 


-2.97* 


No Reform v. 
Trad .School 


-.70* -.95** 


2.18* 


1.03* 


-1.30- 


-.89* 


2.23- 


2.71* 


HLM-computed SD 


.927 1.774 


.315 


.942 


.443 


1.087 


. 181 


.477 



- p < .10; * p < .05; ** p < .01; *** p < .001 



a. The numbering of these outcomes refers to designations in Figure 1. 



b. HLM results computed with within school adjustments from Table 4: students' course taking in academic 

courses in math and science in high school, minority status, gender, SES, 8th-grade ability, and 8th-grade 
engagement . 



c. All effects are presented are in a standardized effect-size metric. Effects computed by dividing the HLM 
gamma coefficient for each outcome by the school -level standard deviation (SD) that outcome. These SDs are 
shown in the bottom panel of this table. 



32 

o 

ERIC 

wm&tm 

. n . , , ata_ka ai. - — 



— ' u 



Explaining Restructuring Effect*, on Achievement Geins 

30 

Table 4: HLM Between- School Model for Barly and Late Achievement Gains in Science and Mathematical Bf facts of 
School Organisational Vectors (N-9,631 Students in 789 Schools) 





Dependent variable* (a) 






Outcome 1 


Outcome 3 


Outcome 2 


Outcome 4 


Early Gain 


SES -Slope on 


Late Gain 


SES-Slope on 


Grade 8- >10 


8->10 Gain 


Grade 10- >12 


10- >12 Gain 


Science Math 


Science Math 


Science Math 


Science Math 



Baue Effect(b,c) 3.50*** 6.92*** 


2.66* 


1.01** 


4.36*** 


2.73* 


1.63 


2.35* 


A. Demographic and Structural Characteristics 
Average SES .26 .90 .99 


.43 


.95 


.70 


.74 


. 16 


Hi-Minority Enrl . .33 -1.54- 


- .67 


-.03 


1.67 


-1.81- 


. 38 


- .73 


Catholic School .29 1.58- 


- .45 


- .72- 


.45 


.92 


- .50 


-1.97 


NAIS School -.15 -.29 


1.82 


-.20 


.72 


- .19 


-2.55 


1.36 


School Size -.43* -.56* 


1.02* 


.98* 


- . 77* 


- .65* 


1.60* 


78* 


B. Restructuring Effects 
Reatructing v. 

Trad. School .24 .52 


-1.56- 


-1.21 


2 . 01 


.23 


-3 69 


-1.31 


No Reform v. 

Trad. School -.31 -.24 


i 

o 

r-* 


1.35 


-1.82- 


- .66 


4.40- 


.72 


C. social Organisation 
Collective Responsibility 

for Learning .39* .89** 


-1.01* 


- .59* 


1 . 00* 


.40* 


-1.48** 


-1 . 37** 


D. Academic Organization 
Average Number of Math & 

Science Courses .49* .74* 


- . 76 


• .95 


1.04* 


. 37*** 


-2.04- 


-1.34- 


Variability in Math. Science 

Course Taking -.26 -.44 


1 . 17* 


.59* 


• .45 


- . 12 


2 . 30* 


1 . 88* 


Authentic Instructional Practices in 
Science & Math .42* .50*** 


- .69 


- .50 


1.66* 


. 36** 


-3.27- 


43 


Variability in Authentic 

Instruction -.29- -.49- 


1.11** 


1 . 34** 


- 25 


- 59- 


3 . 08 * * 


1.41** 


Academic Press .35** .67 


- . 83* 


-.54 


.56* 


.21 


-1.15* 


■ .71* 



- p £ .10; * p £ .05; ** p * .01; *** p ^ .001 



a. The numbering of these outcomes refers to designations in Figure 1. 

b. All models also include within- school adjustments lO: all variables shown in Table 4: students' course 
taking in academic courses in math and science in high school, minority status, gender, SES, Bth-grade 
ability, and 8th~grade engagement. 

c. All effects are presented are in a standardized effect -size metric, computed by dividing the HLM gamma 
coefficient for aach outcome by the HLM adjusted school- level standard deviation (SD> of that outcome 
These SDs are displayed in the bottom panel of Table 6. 
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Appendix: Construct ion Details of Measures Used in This Study Which Were Mot Described in the Lee and 
ftnith <1»»S> MRS Study 

X. Variables Measured on Students 

A. Achievement Outcomes 

♦ F22XMXRR -- Mathematics IRT-estimated number right (12th grade). 

♦ F22XSIRR -- Science IRT-estimated number right (12th grade). 

B. Demographic Controls 

+ SES, Minority Statu a, Gender as used in the ERS. 

C . Academic Controls 

+ Engagement as used in the ERS . 

o Achievement 

♦ 2-score of sum of BY2XRXRS , BY2XHIRS , BY2XMXRS (8th-grade measures) 

o Cour8etaking in Mathematics and Science Early in High School 

For modeling science gains between 8th and 10th grade (Outcomes 1 and 2). we summed 10th graders' 
self-reports of course taking in mathematics and science. Variables coded 0 • none; l • 0.5 to 1 yoar; 2 
■ 1.5 to 2 years. Thus, the sum represents numbers of semesters of mathematics and science coursework. 



For the HLM 


analyses, 


variable was 


z-scored (M • 0, SD • 1) . 


♦ 


F1S22B 




HOW 


MUCH 


COURSEWORK 


IN 


PRE-ALBEBRA 


♦ 


F1S22C 


-- 


HOW 


MUCH 


COURSEWORK 


IN 


ALGEBRA I 


♦ 


F1S22D 


-- 


HOW 


MUCH 


COURSEWORK 


IN 


GEOMETRY 


♦ 


F1S22E 


-- 


HOW 


MUCH 


COURSEWORK 


IN 


ALGEBRA II 


4 


F1S22F 




HOW 


MUCH 


COURSEWORK 


IN 


TRIGONOMETRY 


♦ 


F1S22G 




HOW 


MUCH 


COURSEWORK 


IN 


PRE- CALCULUS 


♦ 


F1S22H 


-- 


HOW 


MUCH 


COURSEWORK 


IN 


CALCULUS 


♦ 


F1S23B 




HOW 


MUCH 


COURSEWORK 


IN 


PHYSICAL SCIENCE 


♦ 


F1S23C 


-- 


HOW 


MUCH 


COURSEWORK 


IN 


BIOLOGY 


♦ 


F1S23E 


-- 


HOW 


MUCH 


COURSEWORK 


IN 


CHEMISTRY 


♦ 


F1S22F 


.. 


HOW 


MUCH 


COURSEWORK 


IN 


PRINCIPALS OF TECHNOLOGY 


«• 


F1523G 


-- 


HOW 


MUCH 


COURSEWORK 


IN 


PHYSICS 


♦ 


F1S22H 




HOW 


MUCH 


COURSEWORK 


IN 


OTHER SCIENCE 



o CourseeaJcing in Mathematics and Science Over 4 High-School Years 

For modelling gains between 10th and 12th grade (Outcomes 3 and 4), we summed several variables, each of 
which measured the number of Carnegie units (a year-long course) in several courses over high school from 
students' transcripts. The summed variable was then aggregated to the school level as a school mean. For 
HLM analyses, the aggregate was z-scored (M • 0 , SD • 1) . 



♦ 


F2RAL 1_C -- 


CARNEGIE 


UNITS 


IN 


ALGEBRA I 


♦ 


F2RAL2JT -■ 


CARNEGIE 


UNITS 


IN 


ALGEBRA II 


♦ 


F2RGEO__C -- 


CARNEGIE 


UNITS 


IN 


GEOMETRY 


♦ 


F2RTRI _C 


CARNEGIE 


UNITS 


IN 


TRIGONOMETRY 


♦ 


F2RPRE_C -- 


CARNEGIE 


UNITS 


IN 


PFE- CALCULUS 


* 


F2RCALJ: -- 


CARNEGIE 


UNITS 


IN 


CALCULUS 


♦ 


F2REAR_C -- 


CARNEGIE 


UNITS 


IN 


EARTH SCIENCE 


♦ 


F2RB IO_C -- 


CARNEGIE 


UNITS 


IN 


BIOLOGY 


* 


F2 RCHE_C 


CARNEGIE 


UNITS 


IN 


CHEMISTRY 


* 


F2RPHY__C -- 


CARNEGIE 


UNITS 


IN 


PHYSICS 


* 


F2ROSC C -- 


CARNEGIE 


UNITS 


IN 


OTHER SCIENCE COURSES 
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II. Variable Meaaurad on Schoola 

A. Bio graphic and Structural Characteristics 

♦ Average SBS , Minority Concentration, Sector, Sire as used in the BRS . 

B. Measures of School Restructuring (Ueed in MRS study and described in Lee and Smith (1993)). 

C. School Social Organisation 



o Collective Reaponeibility tor Learning 

Variables come from teacher questionnaires . The order in which the variables are listed reflect* the 
item-specific factor loading*. Variables combined into a factor from principal components analysis, with 
reliability (Cronbach's alpha) .81. Composite was aggregated and z-scored (M - 0 , SD - 1) . 



♦ F1T4_5E -- 

♦ F1T4_5A -- 

♦ F1T4_5D -- 

♦ F1T4_5F -- 

♦ FlT4_2J -- 

♦ FlT4_5B -- 

♦ F1T4_2N -- 

♦ F1T4_11F-- 

♦ F1T4_1D -- 

♦ FlT4_lI -- 

♦ F1T4_5C -- 

♦ F1T4 5E -- 



LITTLE I CAN DO TO ENSURE HIGH ACHIEVEMENT (Reversed) 

I CAN GET THROUGH TO THE MOST DIFFICULT STUDENT 
DIFFERENT METHODS CAN AFFECT A STUDENT'S ACHIEVEMENT 
TEACHERS MAKE A DIFFERENCE IN STUDENTS' LIVES 
IT IS A WASTE OF TIME TO DO MY BEST AT TEACHING (Rev) 
TEACHERS RESPONSIBLE, KEEPING STUDENTS FROM DROPPING OUT 
STUDENT ATTITUDES REDUCE ACADEMIC SUCCESS (Reversed) 

I TRY TO CREATE LESSONS SO STUDENTS ENJOY LEARNING 
STUDENTS' SUCCESS, FAILURE DUE TO FACTORS BEYOND ME (Rev) 
STUDENTS ARE INCAPABLE OF LEARNING THE MATERIAL (Rev) 

I CHANGE MY APPROACH IF STUDENT ARENT'T DOING WELL 
STUDENT MISBEHAVIOR INTERFERES WITH MY TEACHING (Rev) 



D , School Academic Organisation 



o Average Academic Coursetaking in Math and Science -- aggregated from student reports of coursetaking, 
described above and z-scored (M » 0 , SD ■ 1 x . 



o Variability in Mathematics, Science Coursetaking 

Standard deviation of student coursetaking, from transcripts. Variable was standardized (M » 0, SD • 1) . 



o Authentic Instruction in Ms theme tics and Science 

Details on the F.asch modelling methods used to construct this measure are available from the authors. 
Items in four categories listed below are drawn from teacher and stident reports about instruction in 
each subject at the sophomore year (first NELS followup) . 



Science Items from Students 



♦ F1S29G 

♦ F1S29F 

♦ F1S29B 

♦ F1S29M 

♦ F1S29D 

♦ F1S29N 
f F1S29E 

♦ F1S29C 

♦ F1S29L 



HOW OFTEN DESIGN AND CONDUCT OWN EXPERIMENTS. PROJECT 
HOW OFTEN MAKE UP OWN SCIENTIFIC PROBLEM, ANALYTIC METHOD 
HOW OFTEN CHOOSE OWN SCIENTIFIC OR PROBLEM TO STUDY 
HOW OFTEN DISCUSS CAREER OPPORTUNITIES IN SCIENCE, TECH. 

HOW OFTEN WRITE UP REPORTS OF LAB OR PRACTICAL WORK 
HOW OFTEN WATCH TEACHER DEMONSTRATE OR LEAD IN EXPERIMENT 
HOW OFTEN USE BOOK OR WRITTEN INSTRUCTIONS TO DO EXPERIMENTS 
HOW OFTEN COPY TEACHERS' NOTES FROM BLACKBOARD 
HOW OFTEN LISTEN TO TEACHER LECTURE IN CLASS 



Mathematics Items from student! 

* F1S32E -- HOW OFTEN USE COMPUTERS 
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♦ F1S32F -- HON OFTEN USE HANDS-ON MATERIALS OR MODELS 

+ P1S32B -- HON OFTEN USE BOOKS OTHER THAN MATH TEXTBOOK 

♦ F1S32H -- HON OFTEN PARTICIPATE IN STUDENT-LED DISCUSSIONS 

♦ P1S32I --HOW OFTEN EXPLAIN MATH WORX ORALLY 

♦ P1S32G -- HON OFTEN USE CALCULATORS 

♦ F1S32D -- HOW OFTEN IX) STORY PROBLEMS OR PROBLEM-SOLVING ACTIVITIES 

♦ P1S32C -- HON OFTEN COPY TEACHERS' NOTES PROM BOARD 

+ P1S32A -- HON OFTEN REVIEW YESTERDAY'S WORK 

Mathematics Items from Teachers 

♦ F1T2_18H — HAVE STDS GIVE ORAL REPORTS 

♦ F1T2_18E -- HAVE STD-LED WHOLE GROUP DISCUSSIONS 

♦ F1T2_18F -- HAVE STDS WORK IN SMALL GROUPS 

♦ F1T2_18C -- USE WHOLE GROUP DISCUSSION 

♦ F1T2_18G -- HAVE STDS COMPLETE INDIVIDUAL ASSIGNMENTS IN CLASS 

♦ F1T2_18D -- HAVE STDS RESPOND ORALLY TO QUESTIONS 

science Items from Teachers 

♦ F1T2_18H -- HAVE STDS GIVE ORAL REPORTS 

♦ F1T2_18F -- HAVE STD-LED WHOLE GROUP DISCUSSIONS 

♦ F1T2_18F -- HAVE STDS WORK IN SMALL GROUPS 

♦ F1T2_18C -- USE WHOLE GROUP DISCUSSION 

♦ F1T2_18G -- HAVE STUS COMPLETE INDIVIDUAL ASSIGNMENTS IN CLASS 

♦ F1T2_18D -- HAVE STDS RESPOND ORALLY TO QUESTIONS 

o Variability in Authentic Instruction 

The ctandard deviation of the Rasch- cons true ted measure of authentic instruction in each school. 
Variable was standardized CM • 0, $b *• 1) . 

o Academic Press » 

Variables* from reports by school principals* were combined into a composite# formed with principal 
components factor analysis and z scored (M - 0 , SD - 1) . Reliability (Cronbach's alpha) of .81. 

♦ F1C93G -- STUDENT MORALE IS HIGH 

♦ F1C93D -- TEACHERS PRESS STUDENTS TO ACHIEVE 

♦ F1C93F -- TEACHER MORALE IS HIGH 

♦ F1C93B -- STUDENTS PLACE HIGH PRIORITY ON LEARNING 

♦ F1C93E •- STUDENTS ARE EXPECTED TO DO HOMEWORK 
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Figure 1. Gains in Mathematics and Science 8th to 12th Grade 



Test Score (Standardized to 8th Grade Test) 




Note: The numbers shown in this figure are derived from a three-level HLM estimation of st idem 
growth in mathematics and science. The raw score for 8th grade, along with the raw gains 
estimated for 10th and 12th grade, are estimated after taking social class, gender ethnicity 
individual coursetaking through 12th grade in academic mathematics and science, and 8th ’ 
grade ability into account. These 8th grade raw scores were then converted to a* 
standardized metric based on the 8th grade test overall mean ( 19.68 for science and 8.25 
for mathematics) and overall standard deviation (4.83 for sci en ce and 12.04 for 
mathematics). The raw gains were then divided by this 8th grade standard deviation and 
added to the 8th grade standardized score in each subject. Therefore, the picture shows 
how much gain students make on average in 10th and 12th grades telative to the initial 
performance in 8th grade. 
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